Systems and Methods for Predictive Augmentation of Vehicle Service Procedures

ABSTRACT

An example method includes receiving a first service procedure including a plurality of procedural steps for servicing a vehicle, identifying at least one procedural step of the plurality of procedural steps to supplement with supplemental service information, receiving information about vehicles sharing one or more attributes with the vehicle, determining at least one piece of supplemental service information to supplement the at least one identified procedural step, and providing a supplemented service procedure comprising the first service procedure with the at least one piece of supplemental service information included with the at least one identified procedural step.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/812,941, entitled “Systems and Methods for Predictive Augmentation ofVehicle Service Procedures”, filed Jul. 29, 2015, the contents of whichare fully incorporated herein for all purposes.

BACKGROUND

Various types of vehicles produced by manufacturers occasionally have tobe repaired. In some cases, a vehicle owner may notice a change in theperformance of a vehicle, prompting the vehicle owner to bring thevehicle to a repair shop to diagnose a problem and potentially repairthe vehicle. In other cases, an electronic control module of the vehiclemay detect a fault and provide a malfunction indication via aninstrument panel of the vehicle. If the vehicle owner notices theindication, the vehicle owner may bring the vehicle to a repair shop forservice. Based on symptoms reported by the vehicle owner and/orautomated error indicators such as diagnostic codes, a repair shopemployee may identify one or more service procedures to perform on thevehicle in order to repair the vehicle. For instance, a serviceprocedure may be followed by a mechanic at the repair shop to replacethe vehicle's brake pads, fix a leaking tire, or service a vehicle's airconditioning system. In some examples, a service procedure stored in adigital format may be accessed by a mechanic at a repair shop with acomputing device, such as a laptop computer, a touchpad, or ahead-mounted display (HMD). The mechanic may then view steps of theprocedure on a display of the computing device while performing thosesteps on the vehicle.

SUMMARY

In one example aspect, a method is provided that includes receiving, ata computing device from a first data source that includescomputer-readable service procedures, a first service procedureincluding a plurality of procedural steps for servicing a vehicle. Themethod further includes identifying, at the computing device, at leastone procedural step of the plurality of procedural steps to supplementwith supplemental service information. The method also includesreceiving, at the computing device from a second data source thatincludes computer-readable vehicle information, information aboutvehicles sharing one or more attributes with the vehicle. The methodadditionally includes determining, at the computing device based on thereceived information, at least one piece of supplemental serviceinformation to supplement the at least one identified procedural step.The method further includes providing, by the computing device forpresentation on a display, a supplemented service procedure comprisingthe first service procedure with the at least one piece of supplementalservice information included with the at least one identified proceduralstep.

In another example aspect, a non-transitory computer-readable medium isprovided having stored therein instructions, that when executed by acomputing device, cause the computing device to perform functions. Thefunctions include receiving, from a first data source that includescomputer-readable service procedures, a first service procedureincluding a plurality of procedural steps for servicing a vehicle. Thefunctions further include identifying, at least one procedural step ofthe plurality of procedural steps to supplement with supplementalservice information. The functions also include receiving, from a seconddata source that includes computer-readable vehicle information,information about vehicles sharing one or more attributes with thevehicle. The functions additionally include determining, based on thereceived information, at least one piece of supplemental serviceinformation to supplement the at least one identified procedural step.The functions further include providing, for presentation on a display,a supplemented service procedure comprising the first service procedurewith the at least one piece of supplemental service information includedwith the at least one identified procedural step.

In a further example aspect, a system is provided that includes one ormore processors, and a non-transitory computer-readable data storagedevice storing computer-readable program instructions. Thecomputer-readable program instructions are executable by the one or moreprocessors to receive, from a first data source that includescomputer-readable service procedures, a first service procedureincluding a plurality of procedural steps for servicing a vehicle. Thecomputer-readable program instructions are further executable by the oneor more processors to identify, at least one procedural step of theplurality of procedural steps to supplement with supplemental serviceinformation. The computer-readable program instructions are additionallyexecutable by the one or more processors to receive, from a second datasource that includes computer-readable vehicle information, informationabout vehicles sharing one or more attributes with the vehicle. Thecomputer-readable program instructions are also executable by the one ormore processors to determine, based on the received information, atleast one piece of supplemental service information to supplement the atleast one identified procedural step. The computer-readable programinstructions are further executable by the one or more processors toprovide, for presentation on a display, a supplemented service procedurecomprising the first service procedure with the at least one piece ofsupplemental service information included with the at least oneidentified procedural step.

In still another aspect, a system is provided that includes means forreceiving, at a computing device from a first data source that includescomputer-readable service procedures, a first service procedureincluding a plurality of procedural steps for servicing a vehicle. Thesystem further includes means for identifying, at the computing device,at least one procedural step of the plurality of procedural steps tosupplement with supplemental service information. The system alsoincludes means for receiving, at the computing device from a second datasource that includes computer-readable vehicle information, informationabout vehicles sharing one or more attributes with the vehicle. Thesystem additionally includes means for determining, at the computingdevice based on the received information, at least one piece ofsupplemental service information to supplement the at least oneidentified procedural step. The system further includes means forproviding, by the computing device for presentation on a display, asupplemented service procedure comprising the first service procedurewith the at least one piece of supplemental service information includedwith the at least one identified procedural step.

The foregoing summary is illustrative only and is not intended to be inany way limiting. In addition to the illustrative aspects, embodiments,and features described above, further aspects, embodiments, and featureswill become apparent by reference to the figures and the followingdetailed description.

BRIEF DESCRIPTION OF THE FIGURES

FIGS. 1A, 1B, and 1C illustrate example system components, according toan example embodiment.

FIG. 2 is a block diagram of a method, according to an exampleembodiment.

FIG. 3 illustrates a vehicle service procedure with supplemental serviceinformation according to an example embodiment.

FIGS. 4A, 4B, 4C, and 4D are conceptual illustrations of features of adisplay interface of a computing device, according to an exampleembodiment.

FIG. 5 is a flow chart of an example method for using keywords todetermine supplemental service information, according to an exampleembodiment.

FIG. 6 is a flow chart of an example method for using failure trends todetermine supplemental service information, according to an exampleembodiment.

FIG. 7 is a functional block diagram illustrating an example computingdevice used in a computing system that is arranged in accordance with atleast some embodiments described herein.

FIG. 8 is a schematic illustrating a conceptual partial view of anexample computer program product that includes a computer program forexecuting a computer process on a computing device, arranged accordingto at least some embodiments described herein.

DETAILED DESCRIPTION

The following detailed description describes various features andfunctions of the disclosed systems and methods with reference to theaccompanying figures. In the figures, similar reference numbers identifysimilar components, unless context dictates otherwise. The illustrativesystem and method embodiments described herein are not meant to belimiting. It may be readily understood that certain aspects of thedisclosed systems and methods can be arranged and combined in a widevariety of different configurations, all of which are contemplatedherein.

The example embodiments are applicable to a variety of repairable items,such as a vehicle or some other type of repairable machine or device.For purposes of this description, a vehicle is a mobile machine that maybe used to transport a person, people, or cargo. As an example, anyvehicle described herein may be driven or otherwise guided along a path(e.g., a paved road or otherwise) on land, in water, or in the air orouter space. As another example, any vehicle described herein may bewheeled, tracked, railed, or skied. As yet another example, any vehicledescribed herein may include an automobile, a motorcycle, a light-dutytruck, a medium-duty truck, a heavy-duty truck, a semi-tractor, or afarm machine. As still yet another example, any vehicle described hereinmay include or use any appropriate voltage or current source, such as abattery, an alternator, a fuel cell, and the like, providing anyappropriate current or voltage, such as about 12 volts, about 42 volts,and the like. As still yet another example, any of the vehiclesdescribed herein may include or use any desired system or engine. Thosesystems or engines may include items that use fossil fuels, such asgasoline, natural gas, propane, and the like, electricity, such as thatgenerated by a battery, magneto, fuel cell, solar cell and the like,wind and hybrids or combinations thereof. As still yet another example,any vehicle described herein may include an electronic control unit(ECU), a data link connector (DLC), and a vehicle communication linkthat connects the DLC to the ECU.

Within examples, methods and devices are described for providing asupplemented vehicle service procedure (e.g., for use by a technician ata repair shop). More specifically, a computing device may receive aparticular vehicle service procedure that includes a number of steps tobe performed on a vehicle in order to complete the procedure. Vehicleservice procedures may be received in a digital format from a variety ofdifferent sources. For instance, the service procedure may be anoriginal equipment manufacturer (OEM) procedure for a particular type ofvehicle or vehicle component. The computing device may identify certainsteps of the vehicle service procedure to augment with supplementalinformation. More specifically, the supplemental information may beinformation predicted to assist a technician during performance of theservice procedure. Additionally, the supplemental information may bedisplayed to the mechanic (e.g., on a handheld device or a wearableportable device such as an HMD or a wrist display) at chosen pointsduring performance of the procedure or particular procedural steps.

In some examples, a service procedure for a particular vehicle may beidentified by a computing device based on vehicle symptom informationreported by a vehicle owner and/or an automated system of the vehicle.For instance, the symptom information could be “engine hesitates ontakeoff” or “engine is overheating.” In other examples, the symptominformation could include one or more diagnostic trouble codes (DTCs). ADTC may be a generic or manufacturer-specific code that is used toidentify vehicle problems, such as “Throttle/Petal PositionSensor/Switch Malfunction” or “Transmission Control System Malfunction.”In further examples, the symptom information can include an indicationthat a malfunction indicator lamp of the vehicle is illuminated. Aservice procedure may be selected by a computing device to be performedby a technician in order address identified symptoms. In other examples,a particular service procedure may be manually selected by thetechnician instead. For instance, a technician may request a particularservice procedure (e.g., “Replace Brake Pads”) for a particular type ofvehicle (e.g., a 2003 Toyota Corolla).

Once a service procedure is received, supplemental vehicle informationfor the service procedure may be obtained from a different data sourcethan the data source which provided the service procedure. In oneembodiment, a service procedure may be supplemented by a server incommunication via a communication network with both a vehicle serviceprocedures database and a separate supplemental vehicle informationdatabase. The server may determine which steps of a procedure from thevehicle service procedures database to supplement, and with whichparticular pieces of information from the vehicle information database.

The vehicle information database may contain information about vehiclesthat share one or more attributes with the vehicle being serviced. Forinstance, the vehicle information database may include technicalspecifications about vehicles with particular attributes, such as aparticular year, make, model, and engine type (YMME). In other examples,the particular attributes may include a particular year, make, and model(YMM), or a particular year, make, model, engine type, and system(YMMES). In further examples, the vehicle information database caninclude trend data gathered from one or more repair shops regarding pastrepairs performed on vehicles that share one or more common attributeswith a given vehicle. The attributes can also include other types ofinformation that differentiates different models or types of vehicles,such as regular production option (RPO) codes that refer to a specificoption or modification to a vehicle. For example, the vehicle attributesmay indicate that a vehicle includes a trailer hauling package orvarious other components that a vehicle can be manufactured or equippedwith. In additional examples, the vehicle attributes may also includevehicle usage information, such as ranges of vehicle mileage orgeographical location.

In some examples, supplemental service information may include one ormore metrics associated with a particular vehicle component or systeminvolved in one or more procedural steps. For instance, a computingdevice may identify that a particular step of a procedure involvesadding a fluid to a vehicle component or system. The computing devicemay further determine that the step of the service procedure lackscertain information about the component or system that would be usefulfor a technician performing the step, such as a fluid capacity and/orfluid specifications (e.g., indicating a type of fluid to use such astype F transmission fluid or Textron transmission fluid). Thisadditional information may be pulled from a separate data source andthen added to the procedural step as part of a supplemented serviceprocedure that is displayed to a technician performing the procedure.

In additional examples, supplemental service information may includeother types of information related to particular vehicle components orsystems. For example, a computing device may identify a step of aprocedure that involves application of torque to one or more fastenersof a vehicle component (e.g., indicating to torque a cylinder head to anengine block or a wheel to a hub). A technician's view can then beaugmented with a torque specification and/or torque pattern at anappropriate time to assist the technician in performing the identifiedstep.

As another example, the supplemental service information may include alocation of a vehicle component within the vehicle. For instance, if aparticular procedural step involves disconnecting a radiator hose, thesupplemental information may be an indication that the location at whichto disconnect the radiator hose is at the front of the intake manifold.This type of positional information may be displayed to a technician ina variety of ways. For instance, if the technician is viewing thesupplemented service procedure on an HMD, the particular location may behighlighted or pointed to by a virtual arrow within an augmented realitypresentation. Positional information may be displayed in other ways aswell (e.g., as textual information or as part of a visual image of a 3Dmodel of the vehicle).

In some examples, in order to identify a particular step of a procedureto supplement, one or more terms listed in the step may be matched toone or more terms of an interpretation rule. More specifically, theinterpretation rule may be a mapping from particular terms listed inprocedural steps to particular types of supplemental information toretrieve. The particular teams may be previously identified and storedas keywords to search for within vehicle service procedures. Forinstance, certain terms of an interpretation rule may be actions to beperformed by a technician in performance of a procedure, such as to“fill,” “drain,” or “remove” a certain vehicle component or system. Theterm “fill” may map to one or more related types of information, such asa fluid capacity. The computing device may use the interpretation ruleto identify fluid capacity as a particular type of supplementalinformation to include, and then may retrieve the fluid capacity for theparticular vehicle component or system from a vehicle informationdatabase. Other types of keywords may be included within aninterpretation rule as well or instead. For instance, names ofparticular vehicle systems or components that may be listed inprocedural steps could be mapped to particular types of supplementalvehicle information as well.

In further examples, a computing device may determine when to displayparticular pieces of supplemental information based on one or more inputsignals received during performance of a procedure. In some examples,the input signals may be verbal inputs and/or physical action inputsthat enable a technician to progress through a repair procedure. Forinstance, a specific verbal command or movement may cause a portabledisplay to display a particular piece of supplemental information and/ora particular procedural step. For instance, the display may indicatethat torque specifications for a fastener involved in a particularprocedural step are available, and provide a technician with the optionto select or view the specifications. Additional user commands may allowthe user to change the view of certain supplemental information, such asby zooming in or changing a perspective. User commands may also allow auser to select particular procedural steps (e.g., the previous step orthe next step), or select an entirely different repair procedure.

In additional examples, a supplemented service procedure may be providedto a technician via a portable display with a camera. For instance, anHMD with a camera may be used to visually present virtual indications ofsupplemental information overlaid on top of a video feed from the HMD.As a procedure is performed, video data from the camera may be used todetermine when to display particular pieces of supplemental informationand/or particular procedural steps. For instance, the video data may beprocessed to determine which procedural step a technician is currentlyon, or what particular action a technician is currently taking. Triggerevents may be identified from the video data to trigger the display ofparticular service information. For instance, when the technician isabout to disassemble a particular vehicle component (e.g., a doorpanel), supplemental disassembly information may be displayed to assistthe technician (e.g., types of fasteners and where to pull, twist, orotherwise unfasten particular fasteners to avoid breaking the door panelor fasteners).

In further examples, a technician's view during performance of aprocedure can be augmented with a labor time counter. More specifically,a particular service procedure may be supplemented with an amount oftime to complete the procedure. This amount of time may be used toprovide a technician with an indication of whether he is on pace tocomplete the procedure within the allotted amount of time. For instance,the counter may indicate the time elapsed relative to the providedamount of time, or an amount of time remaining (e.g., as a countdownclock). To determine this amount of time, past completion times for theprocedure may be accumulated and combined (e.g., averaged). The pastcompletion times may be restricted to particular types of vehicles(e.g., YMME slices), locations (e.g., one or more repair shops orgeographical regions), and/or time periods. In additional examples,separate completion times for individual steps may be determined andincluded as supplemental information as well or instead. For instance, atechnician may be presented with a separate countdown clock for eachstep of a procedure.

In additional examples, certain vehicle usage information, such asvehicle mileage and geography, may be received by a computing device andused to determine one or more pieces of supplemental information toinclude within an augmented vehicle service procedure. Further, trenddata may be determined based on past repairs performed on vehicles withone or more common attributes. For instance, one or more vehiclecomponents may be identified for inspection by a technician based on thecurrent mileage of the vehicle approaching a point at which thecomponents may be predicted to fail based on past failures in othervehicles. Additionally, the one or more components may be locatedproximate to a vehicle component already involved in the procedure suchthat it may be efficient for a technician to inspect the components atthe same time. As a specific example, a technician may remove certainvehicle components while servicing an engine, including a timing belt. Acomputing device may locate warnings about the timing belt, such as anindication that the timing belt is recommended to be replaced at 50,000miles, when the vehicle currently has 39,000 miles. The technician maythen be presented with a notification to inspect the timing belt, andpossibly other relevant information as well such as what types of wearor failures to look for, and how close the current vehicle mileage is tothe mileage limit for the component.

In further examples, prior service history of a particular vehicle mayalso be used to determine one or more pieces of supplemental serviceinformation to include within a supplemented service procedure. Forinstance, a technician's view may be augmented with warnings based onpast service repairs performed on the vehicle at the same repair shopand/or at other repair shops (e.g., service records). As a specificexample, a transmission seal on a vehicle may have been nicked duringinstallation of a clutch shaft during a past repair procedure performedon the vehicle. A technician may be provided with a warning to becareful with this part to avoid exacerbating the problem.

In additional examples, a vehicle service procedure may also besupplemented with notifications received during performance of theprocedure. For instance, the procedure may require one or morereplacement parts to be installed in the vehicle. A technician's viewmay be supplemented with notifications from a repair shop partsdepartment indicating when certain replacement parts are available. Insome examples, these notifications may be provided to a technician oneor more steps before the parts are actually needed to perform theprocedure. By providing advance notice, a technician may be better ableto plan out his schedule, which may be particularly helpful when thetechnician is performing a number of procedures on several differentvehicles in parallel. As another example, a technician's view may beaugmented to indicate when a customer has given approval to proceed witha particular procedure or procedural step. A customer approvalnotification may also be provided to a technician one or more stepsbefore a particular procedural step that the customer approval affirms.

As previously noted, a technician may view an augmented serviceprocedure using a variety of computing devices, including touchpaddevices, laptops, wearable computing devices (e.g., glasses, goggles,wrist displays), and other types of portable displays. In some examples,the technician's view can also be augmented with data obtained from avehicle scan tool. Additionally, a portable wearable display cancommunicate via multiple protocols (e.g., wifi and Bluetooth), and canalso link with multiple devices simultaneously (e.g., a remote serverproviding a supplemented vehicle procedure and a diagnostic vehicle scantool). Other types of devices and combinations of devices may also beused to provide and/or display augmented vehicle service procedures aswell.

As mentioned above, this description describes several exampleembodiments. Within this description, the articles “a” or “an” are usedto introduce elements of the example embodiments. The intent of usingthose articles is that there is one or more of the elements. The intentof using the conjunction “or” within a described list of at least twoterms is to indicate any of the listed terms or any combination of thelisted terms. The use of ordinal numbers such as “first,” “second,”“third” and so on is to distinguish respective elements rather than todenote a particular order of those elements. The ordinal numbers can bewritten in the form of 1st, 2nd, 3rd, and so on.

Additional functionalities and examples of the described methods andcomputing devices are also described hereinafter with reference to theaccompanying figures.

Referring now to the Figures, FIG. 1A illustrates an example system 100.As shown in FIG. 1, the example system 100 includes a computing device102, a vehicle service procedures database 110, a vehicle informationdatabase 112, a communication network 114, and a client device 122. Theclient device 122 can represent any type of mobile or stationarycomputing device. By way of example, the client device 122 can be acomputer that is located at a repair shop. However, the example is notmeant to be limiting. In other instances, the client device 122 can be alaptop computer, tablet device, cellular phone, wearable computingdevice, including an HMD, glasses, goggles, a smartwatch or other wristdisplay, or another type of computing device.

The computing device 102 can similarly be any type of computing device.In one instance, the computing device 102 can be a server or a componentof a server that is located remotely from the client device 122 (e.g.,in a cloud computing environment). For example, the computing device 102can be a desktop computer, workstation, or other type of computingdevice configured to operate within a client-server architecture. Inanother instance, the computing device 102 can be a computing devicethat is located at a repair shop. For example, the computing device 102may be a computing device used by a repair shop to manage repair ordersand/or shop equipment that is used to service vehicles. Otherconfigurations are also contemplated.

The computing device 102 can include a memory 104 and a processor 106.Similarly, the client device 122 can also include a memory 124 and aprocessor 126. The processor 106 and the processor 126 can be any typeof processors, such as a microprocessor, digital signal processor,multicore processor, etc., coupled to the memory 104 and memory 124respectively. The memory 104 and the memory 124 can be any type ofmemories, such as volatile memory like random access memory (RAM),dynamic random access memory (DRAM), static random access memory (SRAM),or non-volatile memory like read-only memory (ROM), flash memory,magnetic or optical disks, digital versatile disk read-only memory(DVD-ROM), or compact-disc read-only memory (CD-ROM), among otherdevices used to store data or programs on a temporary or permanentbasis.

Additionally, the computing device 102 can include a communicationinterface 108 and the client device 122 can include a communicationinterface 128. The communication interface 108 and the communicationinterface 128 can facilitate communication with the communicationnetwork 114 for the computing device 102 and the client device 122according to one or more wired or wireless communication standards orprotocols. For instance, the communication interface 108 and thecommunication interface 128 can facilitate data communication over oneor more network links. The communication interface 108 and/or thecommunication interface 128 can include a transmitter to transmit dataand a receiver to receive data. Alternatively, the communicationinterface 108 and/or the communication interface 128 can include atransceiver configured to transmit and receive data.

The client device 122 may also include a display 130. The display 130could be any type of electronic visual display capable of presentinginformation visually to a technician. For instance, the display 130could be capable of presenting visual information in a variety of ways,including text, two-dimensional visual images and/or three-dimensionalvisual images. In additional examples, the display 130 may be capable ofpresenting information to a technician in other ways (e.g., audio,haptics) as well or instead. Example displays include computer monitors,touchscreens, HMD's, or wrist displays. In some examples, the display130 may be capable of displaying an augmented reality presentation inwhich the display shows a presentation of images or video taken of avehicle during performance of a procedure with virtual annotationsoverlaid on the presentation to show additional supplementalinformation. Other types of electronic displays may also be used.

The vehicle service procedures database 110 can be or include a datastorage device, such as a non-transitory computer-readable storagedevice. Although the vehicle service procedures database 110 is shownconnected to the computing device 102 via the communication network 114,in another embodiment, the vehicle service procedures database 110 canbe directly connected to or a component of the computing device 102. Thevehicle service procedures database 110 can store a plurality of vehicleservice procedures, which may be OEM procedures provided bymanufacturers of different types of vehicles or vehicle components. Eachservice procedure can include a plurality of steps pertaining to aparticular type of vehicle repair. In some examples, multiple vehicleservice procedures databases may be accessible by computing device 102.For instance, each vehicle service procedures database may store serviceprocedures pertaining to particular vehicle components or systems.Additionally, different vehicle service procedures databases may storeservice procedures pertaining to different types of vehicles differentmakes, models, years, etc.) as well or instead.

The vehicle information database 112 can be or include a data storagedevice, such as a non-transitory computer-readable storage device.Although the vehicle information database 112 is shown connected to thecomputing device 102 via the communication network 114, in anotherembodiment, the vehicle information database 112 can be directlyconnected to or a component of the computing device 102. Additionally,while vehicle information database 112 and the vehicle serviceprocedures database 110 are illustrated here as separate databases, theycould in other embodiments be separate sections of a single database, orotherwise included in a single non-transitory computer-readable medium.The vehicle information database 112 can store vehicle information(e.g., component metrics) for vehicles with different sets of attributesYMME). In some examples, multiple vehicle information databases may beaccessible by computing device 102. For instance, separate vehicleinformation databases may store information pertaining to differentvehicle components or systems. Additionally, separate vehicleinformation databases may store information pertaining to vehicles withdifferent attributes as well or instead.

In additional examples, the vehicle information database 112 may includetrend data for different types of vehicles as well or instead. The trenddata may be determined based on an accumulation of past repairprocedures performed by one or more repair shops. The vehicleinformation database 112 may also directly store repair orders withinformation about past vehicle repairs as well or instead.

The computing device 102 may be capable of searching the vehicleinformation database 112 for particular pieces of supplemental serviceinformation. In some instances, the vehicle information database 112 canalso store search queries and associated search results from priorsearches to facilitate quickly retrieving information from the vehicleinformation database 112. For instance, the computing device 102 can beconfigured to access a list of prior searches before searching theentire vehicle information database 112 to identify whether searchresults are already stored by the vehicle information database 112 for agiven search. The stored search results may also be updated on aperiodic basis by the vehicle information database 112 (e.g., when newrepair orders or other vehicle information is received).

FIG. 1B illustrates an example workflow between the system components ofFIG. 1A. More specifically, an example scenario is provided in whichclient device 122 receives with a supplemented service procedure 140from computing device 102. Initially, the client device 122 may reportcertain information about a vehicle to be repaired to computing device102, which may include a vehicle type 132 and vehicle symptoms 134. Thevehicle type 132 may include vehicle attributes (e.g., year, make,model, engine type) as well as usage information (e.g., mileage,geography). The vehicle symptoms 134 may be one or more conditionsexhibited by the vehicle and/or reported by a vehicle owner such as“engine hesitates on takeoff,” “rough engine idle,” “engine cranks butwon't start,” “poor gas mileage,” or “check engine light is on.” A checkengine light can also be referred to as a malfunction indicator lampthat, when illuminated on an instrument panel of a vehicle, indicates amalfunction of a computerized engine management system of the vehicle.Alternatively or in addition, the vehicle symptoms 134 may include oneor more DTCs and/or an indication that a malfunction indicator lamp ofthe vehicle is illuminated.

Based on the vehicle type 132 and the vehicle symptoms 134, thecomputing device 102 may identify at least one service procedure 136from vehicle service procedures database 110 for servicing the vehicle.In some examples, the service procedure 136 may be an OEM procedureprovided in an electronic format by a manufacturer of the vehicle type132 and/or specific vehicle components. For instance, the serviceprocedure 136 may include steps to locate and replace a particularelectronic sensor contained within the engines of vehicles of vehicletype 132. In some examples, the service procedure 136 may beautomatically selected by computing device 102 and/or client device 122based on vehicle type 132 and vehicle symptoms 134. In other examples,the service procedure 136 may be manually selected by a technician usingclient device 122 via a user input interface of client device 122.Service procedure 136 may be identified from data source 110 in otherways as well.

After receiving service procedure 136, computing device 102 may processservice procedure 136 to identify one or more steps of service procedure136 to augment with supplemental service information 138. At least someof the supplemental service information 138 may be obtained from vehicleinformation database 112, which may include information describingvehicles sharing one or more attributes (e.g., YMME) with a vehicle ofvehicle type 132. In further examples, supplemental service information138 may be obtained from one or more other sources as well or instead.

The supplemental service information 138 may be incorporated within oneor more steps of service procedure 136 by computing device 102 toproduce supplemented service procedure 140. Supplemented serviceprocedure 140 may be provided by computing device 102 in an electronicformat that can be displayed by client device 122 on display 130. Forinstance, client device 122 may then provide visual, audio, and/or otherindications of the supplemental service information 138 while thetechnician performs corresponding steps of the supplemented serviceprocedure 140 on the vehicle.

FIG. 1C illustrates several example client devices. More specifically,computing device 102 may communicate with one or more client devicesthrough communication network 114 via one or more respectivecommunication links. For instance, computing device 102 may communicatewith laptop computer 122A via communication link 152A, HMD 122B viacommunication link 152B, and/or touchpad device 122C via communicationlink 152C. Computing device 102 may transmit information to one or moreof client devices 122A, 122B, and/or 122C all at once, or computingdevice 102 may transmit information periodically. In some examples,computing device 102 may transmit and/or receive certain pieces ofinformation to and/or from one or more of client devices 122A, 122B,and/or 122C while a technician performs a service procedure on a vehiclewith the assistance of one or more of the client devices. In furtherexamples, additional client devices may be used to receive and display asupplemented service procedure than those specifically illustrated hereas well.

Turning now to FIG. 2, FIG. 2 is a block diagram of an example method200 for providing a supplemented service procedure. Method 200 shown inFIG. 2 presents an embodiment of a method that could be used orimplemented by computing device 102 and/or client device 122 of FIG. 1,for example, or by components of computing device 102 and/or clientdevice 122, or more generally by any of a variety of computing devices.Method 200 can include one or more operations, functions, or actions asillustrated by one or more of blocks 202-210. Although the blocks areillustrated in a sequential order, these blocks can also be performed inparallel, and/or in a different order than those described herein. Also,the various blocks can be combined into fewer blocks, divided intoadditional blocks, and/or removed based upon the desired implementation.

In addition, for the method 200 and other processes and methodsdisclosed herein, the block diagram shows functionality and operation ofone possible implementation of present embodiments. In this regard, eachblock can represent a module, a segment, or a portion of program code,which includes one or more instructions executable by a processor orcomputing device for implementing specific logical functions or steps inthe process. The program code can be stored on any type ofcomputer-readable medium, for example, such as a storage deviceincluding a disk or hard drive. The computer-readable medium can includenon-transitory computer-readable medium, for example, such ascomputer-readable media that stores data for short periods of time likeregister memory, processor cache and random access memory (RAM). Thecomputer-readable medium can also include non-transitory media, such assecondary or persistent long term storage, like read only memory (ROM),optical or magnetic disks, compact-disc read only memory (CD-ROM), forexample. The computer-readable medium can also be any other volatile ornon-volatile storage systems. The computer-readable medium can beconsidered a computer-readable storage medium, for example, or atangible storage device.

In addition, for the method 200 and other processes and methodsdisclosed herein, each block in FIG. 2 can represent circuitry that iswired to perform the specific logical functions in the process.

Initially, at block 202, the method 200 includes receiving a firstservice procedure that includes a plurality of steps for servicing avehicle. More specifically, the first service procedure may be receivedby a computing device from a data source that includes computer-readableservice procedures, such as OEM procedures. In some examples, the firstservice procedure may be received at a point in time before a vehicle isbrought in for service. In particular, the service procedure may besupplemented in advance for one or more different types of vehicles. Inother examples, the first service procedure may be received in responseto a request from a client device, such as an HMD used by a technicianwhile servicing a particular vehicle. In further examples, the firstservice procedure may be received from the client device itself or fromanother source.

FIG. 3 illustrates an example vehicle service procedure. Morespecifically, thermostat replacement procedure 300 may include steps302-316 for replacing an engine thermostat in a particular type ofvehicle, such as a 2007 Chevy Cobalt. Each block 302-316 may represent asection of computer-readable medium accessible by a client device todisplay each of the steps 302-316. In sonic examples, each of the blocks302-316 may include a module, a segment, or a portion of program code,which includes one or more instructions executable by a processor orcomputing device to provide a presentation of the steps 302-316, thedisplay of which may include various types of visual, auditory, and/orother types of output. Various types of information may be includedwithin blocks 302-316, and made accessible to a technician duringperformance of the procedure. Other vehicle service procedures mayinclude fewer or more steps than thermostat replacement procedure 300illustrated here. Additionally, other types of supplemental serviceinformation than those specifically illustrated within this example maybe included within a supplemented service procedure as well. Further, avehicle service procedure may apply to various types of vehicles,including, for example, multiple years of a particular model, ormultiple models of a particular make.

Referring back to FIG. 2, method 200 may further involve identifying atleast one procedural step to supplement with supplemental serviceinformation, as shown by block 204. More specifically, the computingdevice may process the steps of the received procedure to identify wheresupplemental information may be inserted to assist a technician inperformance of the procedure. In some examples, this step may involveidentifying particular vehicle components or systems involved in theprocedure for which supplemental information is available. Thecomponents may be specifically listed in the steps or determined basedon the functions to be performed that are instructed by the steps. Infurther examples, steps may be identified to supplement by searching forone or more keywords within the procedural steps. Steps may beidentified to supplement with supplemental information in other ways aswell. In some examples, identifying steps to supplement may furtherinvolve determining that a provided procedure or procedural step doesnot already include certain types of information before retrieving theinformation for a particular vehicle to add as supplemental information.

In reference to FIG. 3, a computing device may identify several steps ofthermostat replacement procedure 300 to supplement with supplementalinformation. In particular, steps 302, 304, 306, 312, 314, and 316 maybe identified as steps to add one or more pieces of supplementalinformation. Additionally, steps 308 and 310 may not be identified assteps to add supplemental information. Each of the steps 302, 304, 306,312, 314, and 316 to supplement may be identified at a single time, orsome of the steps may be identified at different times, possibly duringperformance of the procedure. Additionally, the respective pieces ofsupplemental information for each of the steps 302, 304, 306, 312, 314,and 316 may be determined at the same time that the steps areidentified, or at a later time. In some examples, certain types ofinformation to include with particular steps may first be identified,and then one or more vehicle information sources may be queried to findinformation of the chosen types for a particular vehicle type.

One or more of the steps 302, 304, 306, 312, 314, and 316 may beidentified to augment based on one or more components involved in thesteps. The components may be listed by name within the procedural stepsand/or may be inferred based on the operations to be performed in thesteps. For example, step 304 may be identified because it involves aradiator hose, step 306 and/or step 312 may be identified because eachinvolves a thermostat housing, and/or step 314 may be identified becauseit involves a coolant hose. Additionally, one or more of the steps 302,304, 306, 312, 314, and 316 may be identified based on one or morekeywords listed in the steps. In some examples, the keywords maycorrespond to actions to be performed by a technician in performing theprocedure. For instance, step 302 may be identified based on the keyword“drain” and/or step 316 may be identified based on the keyword “refill.”In additional examples, a combination of components involved and actionsto be performed may be used to identify steps to augment.

In further examples, one or more of the steps 302, 304, 306, 312, 314,and 316 may be identified to augment in other ways as well or instead.For instance, a mapping between types of operations and types ofsupplemental information may be predetermined. The mapping may then beapplied to different procedures to identify steps involving operationsfor which supplemental information should be retrieved. In someexamples, the types of operations in the mapping may be identified bylabor operation codes (LOC's) that refer to particular operations thatcan be performed to a given vehicle. The LOCs can conform to thosedefined by a vehicle manufacturer, a service provider that generates arepair order, a service information provider, or some other entity.

Referring back to FIG. 2, the method 200 may further involve receivinginformation about vehicles sharing one or more attributes with thevehicle, as shown by block 206. More specifically, the computing devicemay request particular types of vehicle information from at least onesupplemental data source that is separate from the first data sourcewhich provided the service procedure. The one or more attributes mayinclude any vehicle characteristics that may signify a type of vehicle,which may include, for example, year, make, model, engine type, otherspecific component types, other design options, vehicle mileage, and/orgeographic history. In some examples, the supplemental data source maybe queried for particular types of information (e.g., metrics,specifications, or locations of components) that relate to the type ofvehicle to be serviced. In further examples, different common attributesmay be used to indicate related vehicles based on the particular typesof information sought. Additionally, multiple separate supplemental datasources may be queried to obtain different types or pieces ofinformation.

The method 200 may further involve determining at least one piece ofsupplemental service information to supplement the at least oneidentified procedural step, as shown by block 208. More specifically,the received information about vehicles sharing one or more commonattributes with a particular vehicle may be processed to identifysupplemental service information to include with steps of the serviceprocedure. In some examples, certain pieces of received vehicleinformation may be directly added to augment procedural steps. In otherexamples, the received vehicle information may first undergo someprocessing or transformation. For example, the received vehicleinformation may indicate where bolts are located on a particular vehiclecomponent. Based on this information, a computing device may determinehow a technician should position a wrench to remove the bolts, and thewrench positions may then be visually displayed as supplemental serviceinformation. In other examples, a computing device may search thesupplemental data source to see what information is available (e.g.,related to a particular vehicle component), and then select particularinformation to include within a supplemented procedure based on whatinformation is available and what operations the procedural stepinvolves.

In reference to FIG. 3, several examples of supplemental serviceinformation are illustrated for steps of thermostat replacementprocedure 300. For instance, step 302 which indicates to drain enginecoolant may be supplemented with supplemental information 322 indicatingto drain from lower radiator hose. In this example, the supplementalinformation 322 indicates to a technician a particular vehicle component(e.g., the lower radiator hose) on which to perform an operation withinthe vehicle. The supplemental information 322 may additionally indicatewhere the lower radiator hose is within the vehicle.

In additional examples, other pieces and/or types of supplementalinformation may be added to step 302 as well or instead. Another exampleclass of supplemental information includes safety precautions orwarnings. For instance, step 302 may be supplemented with a warningindicator, such as “Warning: Engine coolant may be very hot andpressurized. Release cooling system pressure by turning radiator capone-quarter turn counter-clockwise.” In further examples, warningsignals may correspond to particular vehicle components, particulartypes of operations, or a combination of factors.

As another example, step 304 which indicates to disconnect the radiatorhose may be supplemented with supplemental information 324 indicating toperform the operation at the front of the intake manifold. In thisexample, the supplemental information 324 includes a positionalreference indicating a location (e.g., at the front of the intakemanifold) within a particular type of vehicle at which a technicianshould perform an operation (e.g., disconnect the radiator hose). Thesupplemental information 324 may be different for other types ofvehicles (e.g., a vehicle for which this operation is to be performed ata different location).

As a further example, step 306 which indicates to remove thermostathousing may be supplemented by supplemental information 326 identifyingtwo bolts with 13 millimeter bolt heads. In this example, thesupplemental information 326 provides metrics related to fasteners thatmust be removed by a technician to perform the step. For instance, thisinformation may be used by a technician to select a tool to remove thebolts. The supplemental information 326 may include additionalinformation related to how to remove the thermostat housing as well,such as where the bolts are located in a particular vehicle and/or howto position a wrench to remove the bolts.

As another example, step 312 which indicates to install thermostathousing may be supplemented by supplemental information 332, indicatingto torque the bolts to 18 foot-pounds. Additionally, step 314 whichindicates to install the coolant hose may be supplemented bysupplemental information 334, indicating to torque the clamp bolt to 10inch-pounds. In these examples, the respective pieces of supplementalinformation 332 and 334 indicate to a technician how much torque toapply when attaching certain fasteners. Additional torque specificationsmay be provided as supplemental information as well or instead. Torquepatterns indicating a sequencing to apply torque to different fastenersmay also be provided as supplemental information as well or instead.

As a further example, step 316 which indicates to refill engine coolantmay be supplemented with supplemental information 336 indicating to fillwith 13 quarts of mixed dexcool® engine coolant. In this example, thesupplemental information 336 indicates both an amount of fluid and atype of fluid for a technician to use in filling a particular vehiclesystem in further examples, other fluid specifications may be providedas supplemental information for particular vehicle components or systemsinvolved in particular procedural steps as well or instead. As anotherexample, the supplemental information may indicate when and how to add afluid (e.g., to check transmission fluid with the engine running, or tocheck the engine oil when the engine is stopped).

In further examples, one or more pieces of supplemental information maybe provided as additional steps in the supplemented service procedure aswell or instead. For instance, in reference to FIG. 3, supplementalinformation 330 may be added to indicate an additional step to install anew thermostat housing gasket. The supplemental information 330 may beprovided between existing steps 310 and 312. In some examples,supplemental information 330 may be determined based on tips learnedfrom past repair orders or technician notes. For instance, past repairorders may indicate that an OEM procedure omits a particular step, orthat certain types of service procedures sometimes omit particularsteps. In this case, supplemental information 330 may include both anadditional operation to be performed as well as information indicatinghow to perform the operation (e.g., to install the new gasket using agasket sealant with metal or fibrous gasket, not with elastomericgasket).

Referring back to FIG. 2, the method 200 may further involve providing asupplemented service procedure for display as shown by block 210. Morespecifically, the supplemented service procedure may include the stepsof the first service procedure with at least one piece of supplementalinformation included with at least one of the procedural steps. Thesupplemented service procedure may be provided in a digital format thatenables a client device to display the supplemental information as, forexample, text, images, as part of an augmented reality presentation,and/or as auditory feedback. In some examples, the supplemented serviceprocedure may be determined at a point in time before a particular typeof vehicle arrives for servicing. The supplemented service procedure maythen be stored digitally and accessed at a later point in time. In otherexamples, some or all of the supplemental information included in thesupplemented service procedure may be determined after a vehicle of aparticular type arrives for servicing and/or during performance of oneor more steps of the procedure.

In some examples, the supplemented service procedure may includesupplemental information for certain procedural steps as well as one ormore triggers indicating when to display the information duringperformance of the procedure. The triggers may be based on one or moreuser input actions, such as vocal input or movement. For instance, auser may request via a client device to see torque specifications beforeunfastening a particular vehicle fastener, and the torque specificationsmay then be displayed on the client device as supplemental serviceinformation. In further examples, the triggers may be based on videodata of the vehicle (e.g., from an HMD) as the procedure is performed.For instance, the video data may be processed to determine that atechnician is about to apply torque to a particular bolt, and then anamount of torque to apply may be displayed to the technician assupplemental information. In further examples, a technician may also beable to change the view of certain supplemental information (e.g., tozoom in on an image) during performance of the procedure.

As noted, supplemental information may be displayed or otherwisepresented to a technician using a client device in a variety of ways. Insome examples, different modes of presenting information may be useddepending on the type of client device available. For instance,supplemental information may be presented as text on a wrist display, ascomputerized images on a touchpad, or as overlaid virtual annotations onan HMD that is oriented towards the vehicle. Other methods ofcomputerized presentation of information may also be used.

FIGS. 4A, 4B, 4C, and 4D are conceptual illustrations of examplefeatures of a display interface of a computing device. For instance, thedisplay interface can be an interface of a client device that receives asupplemented service procedure and displays procedural steps withsupplemental information to a technician. The display interface may bean interface of any of the types of client devices described herein,including a laptop computer, a stationary computer, a wearable computingdevice including an HMD or a wrist display, a smartphone, or a touchpaddevice. Additionally, while supplemental information is displayed astext here, it could be displayed graphically, as computerized images, oras some other type of presentation.

In reference to FIG. 4A, an example display interface 400 is illustratedfor a client device. A technician can specify certain vehicleinformation 402 via the interface 400, which may be used to determine avehicle service procedure to service a particular vehicle. By way ofexample, the interface 400 displays vehicle information 402 thatincludes a vehicle identifier (2007 Chevy Cobalt), mileage (58,456), anda symptom (Engine Code PO128: Coolant Temperature Below ThermostatRegulating Temperature). In other examples, some or all of vehicleinformation 402 may be determined automatically by a client device(e.g., by receiving identifying information and/or diagnostic codesdirectly from the vehicle).

Based on the vehicle information 402, a service procedure may beselected to service the vehicle, in this case, the selected serviceprocedure is thermostat replacement 404. In some examples, the serviceprocedure may be selected by a technician via the interface 400 (e.g.,by selecting the service procedure from a dropdown list of procedures).In further examples, the selected service procedure may be obtained froma repair order for the vehicle. In other examples, the service proceduremay be automatically determined by the client device and/or by a remoteserver system based on the vehicle information 402. In any case, theclient device may be provided with a supplemented service procedure thatincorporates certain pieces of supplemental information at one or moresteps. At least some of the supplemental information may be determinedbased on vehicle sharing one or more attributes with the vehicle (e.g.,other 2007 Chevy Cobalts).

In addition to identifying the service procedure as thermostatreplacement 404, the display interface 400 may display an amount of time406 to complete the procedure, shown here as 1.5 hours. In someexamples, the amount of time 406 may be provided as supplementalinformation. For instance, the amount of time 406 may be determinedbased on past completion times of the thermostat replacement procedureon other vehicles. The past completion times used to determine theamount of time 406 may be restricted to particular types of vehicles(e.g., YMME slices), locations (e.g., one or more repair shops orgeographical regions), and/or time periods. The amount of time 406 maybe determined by averaging together or otherwise combining the pastcompletion times.

The display interface 400 may additionally contain an instruction 408that allows a technician to give a vocal command (“Begin”) to start theprocedure. In other examples, the first step of the procedure mayautomatically be displayed when the device recognizes that a technicianhas started the procedure. For instance, the device may be an HMD whichreceives video data of the vehicle. When the video data indicates thatthe procedure has begun, the display interface 400 may display the firststep of the procedure along with any supplemental information. Acountdown clock starting from the amount of time 406 may also startbased on the video data indicating that the technician has started theprocedure.

FIG. 4B illustrates an example display of a client device showing afirst step of a supplemented procedure. More specifically, the firststep 410 of the procedure for thermostat replacement 404 may be to drainthe engine coolant. This step may be part of the original serviceprocedure before augmentation. The first step 410 of the procedure maybe augmented (e.g., by a remote server system) to include additionalsupplemental information 412. The supplemental information 412 in thisexample includes an identification of a vehicle component to indicatethat the engine coolant is to be drained from the lower radiator hose.The supplemental information 412 may be displayed in other ways as wellor instead. For instance, an image of the vehicle may be displayed withan arrow pointing to the lower radiator hose. In some examples, thesupplemental information 412 may only be displayed on display 400 inresponse to a user query (e.g., asking where to drain the enginecoolant).

The display 400 may additionally show a remaining amount of time 414counting down from the initial amount of time 406 displayed in FIG. 4A.The remaining amount of time 414 may provide a technician with anindication of how fast he is progressing through the procedure relativeto an expected completion time. In further examples, one or more warningsignals may be provided indicating when the technician is running behinda pace estimated to complete the procedure in the expected amount oftime.

FIG. 4C illustrates another example display of a client device showinganother step of a supplemented service procedure. More specifically, thesecond step 416 of the procedure for thermostat replacement 404 may beaugmented to include additional supplemental information 418. In thisexample, the supplemental information 418 includes a positionalreference to a particular location (e.g., at the front of the intakemanifold) in the vehicle at which to perform the step (e.g., todisconnect the radiator hose)

As shown in FIG. 4C, additional options may be presented to a technicianvia display interface 400 to allow a technician to navigate through aprocedure. In particular, the technician may be able to navigate to theprevious step 420 or the next step 422. Additionally, the technician maybe able to select zoom view 424 to provide a closer view of a portion ofthe vehicle. For instance, in this case, zoom view 424 may provide amagnified view of the location at the front of the intake manifold atwhich to disconnect the radiator hose. Other selectable options may bepresented as well or instead. In some examples, the particularselectable options available for a given procedural step may depend onwhat supplemental service information is available for the step withinthe supplemented service procedure.

FIG. 4D illustrates an additional example display of a client deviceshowing an additional step of a supplemented service procedure. Morespecifically, the fourth step 426 of the service procedure forthermostat replacement 404 may be to remove the engine thermostat. Inthis example, supplemental information 428 may be provided to indicatethat a replacement thermostat is available. A notification may beprovided to the client device during performance of the procedureindicating that the replacement part is available (e.g., from the partsdepartment of a repair shop). The supplemental information 428 thereforemay be determined during performance of the procedure in this case. Asillustrated here, the notification may be provided at a procedural stepone or more steps in advance of when the part is needed (e.g., when thereplacement thermostat will be installed). Advanced notice may assist atechnician in planning out his schedule. Other types of supplementalinformation may be determined during performance of the procedure aswell. For instance, an indication that a customer has approved theprocedure or a certain portion of the procedure may be indicated on thedisplay interface 400 when the customer approval is received (e.g., froma different area of a repair shop).

As shown in FIG. 4D, the display may also display a separate elapsedamount of time 430 for the current procedural step 426 in addition to orinstead of an elapsed amount of time 432 for the full procedure. Theelapsed amount of time 430 for the particular step 426 may be displayedrelative to an expected completion time of the particular step 426,which may be determined based on past completion times of the particularstep 426. For instance, in this example, the display indicates that 4minutes and 35 seconds have elapsed from an expected 10 minutes toremove the engine thermostat, and that 52 minutes have elapsed from anexpected 1 hour and 30 minutes to complete the full thermostatreplacement procedure. The expected time of 10 minutes to complete thestep may be determined based on past step completion times that arerestricted to particular types of vehicles (e.g., YMME slices),locations (e.g., one or more repair shops or geographical regions),and/or time periods. The expected time may be determined by averagingtogether or otherwise combining the past a completion times.

In further examples, the computing device may record an amount of timespent by a technician in performing a service procedure and/orrespective amounts of time to complete particular procedural steps. Insome examples, timing information may only be recorded after receivinguser approval. The recorded time(s) may be stored (e.g., in vehicleinformation database 112 as described in reference to FIG. 1) and usedto determine expected time(s) to complete the procedure or individualprocedural steps on other vehicles in the future.

Turning now to FIG. 5, FIG. 5 is a block diagram of an example method500 for using keywords to determine a supplemented service procedure.Method 500 shown in FIG. 5 presents an embodiment of a method that couldbe used or implemented by the computing device 102 of FIG. 1, forexample, or by components of the computing device 102 of FIG. 1, or moregenerally by any of a variety of computing devices. Method 500 caninclude one or more operations, functions, or actions as illustrated byone or more of blocks 502-512. Although the blocks are illustrated in asequential order, these blocks can also be performed in parallel, and/orin a different order than those described herein. Also, the variousblocks can be combined into fewer blocks, divided into additionalblocks, and/or removed based upon the desired implementation. Each blockcan represent a module, a segment, or a portion of program code, whichincludes one or more instructions executable by a processor forimplementing specific logical functions or steps in the processor. Inaddition, each block in FIG. 5 can represent circuitry that is wired toperform specific logical functions in the process.

Initially, at block 502, the method 500 includes receiving a vehicleservice procedure. The vehicle service procedure may be received by acomputing device in a computer-readable format from a data source thatincludes computer-readable service procedures, and may include aplurality of steps for servicing a vehicle.

At block 504, the method 500 further includes identifying a keywordlisted in a procedural step of the vehicle service procedure. Morespecifically, a predetermined set of keywords may be included in aninterpretation rule that maps the keywords to corresponding types ofvehicle information. The keywords could be textual references to “fill,”“drain,” “remove,” or other operative language that may correspond toinformation that would be of interest to a technician performing theoperation. Different types of keywords may be used as well or instead,such as vehicle component names.

At block 506, the method 500 additionally includes identifying a type ofvehicle information based on the keyword. The keyword may correspond toone or more types of interest that may be of interest to a technicianperforming the step containing the keyword. Additionally, thiscorrespondence may be predetermined as part of an interpretation rule.For instance, “fill” could map to a type of fluid to fill a vehiclecomponent with, while “drain” could map to a location in the vehicle atwhich to drain the fluid from. Particular keywords may map to multipletypes of vehicle information as well.

At block 508, the method 500 also includes searching the vehicleinformation database for the identified type of vehicle information.Once a particular type of vehicle information to include is identified,the information for a particular vehicle may be determined by searchinga database of information about vehicle sharing one or more attributeswith the particular vehicle. For instance, if the identified type ofinformation is a location of a particular vehicle component, thelocation of the particular vehicle component in a particular type ofvehicle (e.g., with a particular YMME) may be retrieved from anappropriate database.

At block 510, the method 500 further includes using the search resultsto determine supplemental service information for the vehicle.Additionally, at block 512, the method also includes supplementing theservice procedure with the determined supplemental service information.Once a relevant type of information is identified based on a keyword andinformation of the relevant type for the particular vehicle isretrieved, the information may be incorporated into the step of thesupplemented service procedure containing the identified keyword. Forinstance, the step of “Drain Engine Coolant” may be supplemented withinformation indicating to drain the engine coolant from the lowerradiator hose in a 2007 Chevy Cobalt.

FIG. 6 is a flow chart of an example method 600 for using vehicle usageinformation to determine a supplemented service procedure. Method 600shown in FIG. 6 presents an embodiment of a method that could be used orimplemented by the computing device 102 of FIG. 1, for example, or bycomponents of the computing device 102 of FIG. 1, or more generally byany of a variety of computing devices. Method 600 can include one ormore operations, functions, or actions as illustrated by one or more ofblocks 602-610. Although the blocks are illustrated in a sequentialorder, these blocks can also be performed in parallel, and/or in adifferent order than those described herein. Also, the various blockscan be combined into fewer blocks, divided into additional blocks,and/or removed based upon the desired implementation. Each block canrepresent a module, a segment, or a portion of program code, whichincludes one or more instructions executable by a processor forimplementing specific logical functions or steps in the processor. Inaddition, each block in FIG. 6 can represent circuitry that is wired toperform specific logical functions in the process.

At block 602, the method 600 includes receiving a vehicle serviceprocedure. The vehicle service procedure may be received by a computingdevice in a computer-readable format from a data source that includescomputer-readable service procedures, and may include a plurality ofsteps for servicing a vehicle. Additionally, at block 604, the method600 includes receiving vehicle usage information for a vehicle that isto undergo the received vehicle service procedure. The vehicle usageinformation may include vehicle mileage, geographical region of use, andpossibly other usage information such as what percentage of thevehicle's use took place on expressways.

At block 606, the method 600 further includes identifying one or morefailure trends based on the vehicle usage information. Morespecifically, failure trends may be identified based on past failures ofvehicles which share certain attributes that make the vehiclesfunctionally similar to the vehicle being serviced. In some examples,the failure trends may be determined based on past repair orders forrelated vehicles and/or recommended services that are identified by amanufacturer. For instance, the failure trends can indicate which partsof a particular vehicle type need to be repaired over time, and anaverage mileage or range of mileage when each part usually needs to berepaired. In further examples, past repair orders for related vehiclesmay be stored in and retrieved from a database, such as vehicleinformation database 112 as described in reference to FIG. 1.

To identify relevant failure trends, the computing device may search forrepair orders matching the particular vehicle type and performed withina range of miles of the odometer reading of a vehicle. To determine therange of miles, the computing device may subtract a predeterminedthreshold number of miles from a current vehicle mileage to determine aminimum and add a predetermined threshold number of miles to the currentvehicle mileage to determine a maximum. If the odometer reading is56,123 miles, for example, the range of miles can be 41,123 miles to71,123 miles. Thus, the computing device can identify repair orders forthe particular vehicle type with a reported mileage of between 41,123miles and 71,123 miles. Alternatively, the computing device may identifywhich of a plurality of predetermined mileage ranges that the mileagecorresponds to. For instance, if the predetermined ranges are 0 to25,000 miles, 25,000 miles to 50,000 miles, 50,000 to 75,000 miles, andso on at 25,000 mile increments, the computing device may determine thatthe mileage range is 50,000 to 75,000 miles.

At block 608, the method 600 may further involve using the failuretrends to determine supplemental service information for the vehicle.More specifically, warnings for specific vehicle components that may belikely to fail can be added as supplemental information for atechnician. The warnings may indicate that the technician should inspectcertain components for certain types of predicted failures, or otherwisepay special attention to particular components.

Additionally, for efficiency, this supplemental information may beincluded for vehicle parts related to a particular service procedure.More specifically, supplemental information may be included for vehiclecomponents proximate to one or more components already involved in theprocedure. As an example, the proximate parts may be parts that are ofthe same system of the vehicle as a particular part being serviced, orthe proximate parts may be parts that are connected to the particularpart. For instance, the particular part may be a clutch, and theproximate parts may include an axle. If it is determined that axles ofthe vehicle type tend to leak near or within a threshold amount of thecurrent vehicle mileage, a supplemental warning may be added to inspectthe axle for leakage. As another example, the particular part may be awater pump and the proximate parts may include a water hose.

As a further example, a technician may be performing a procedure toreplace the brake pads on a car with 89,000 miles. The computing devicemay determine that similar model vehicles are known to have brakecylinders which tend to leak when the vehicle mileage reaches 115,000miles. Supplemental information may therefore include a warning to thetechnician to inspect the brake cylinders while performing the brake padreplacement procedure.

At block 610, the method 600 further includes supplementing the serviceprocedure with the determined supplemental service information. Thesupplemental service information determined based on usage history maybe displayed in various ways. For instance, components with possibleimpending failures may be listed during related procedural steps.Additional information such as expected types of failure and times tofailure may be available to the technician as well. In a furtherexample, an exploded component view of a system (e.g., a component viewof a brake system) can be provided which identifies any relevant failuretrends affecting any components of the system (e.g., any brakecomponents) while a procedure on the system (e.g., brake padreplacement) is performed.

As described above, the computing devices described herein can be anytype of computing device. FIG. 7 is a functional block diagramillustrating an example computing device 700 used in a computing systemthat is arranged in accordance with at least some embodiments describedherein. The computing device 700 can be implemented to determineestimate information or perform any of the functions described abovewith reference to FIGS. 1-8. In a basic configuration 702, computingdevice 700 can typically include one or more processors 710 and systemmemory 720. A memory bus 730 can be used for communicating between theprocessor 710 and the system memory 720. Depending on the desired configuration, processor 710 can be of any type including but not limited toa microprocessor (μP), a microcontroller (μC), a digital signalprocessor (DSP), or any combination thereof. A memory controller 715 canalso be used with the processor 710, or in some implementations, thememory controller 715 can be an internal part of the processor 710.

Depending on the desired configuration, the system memory 720 can be ofany type including but not limited to volatile memory (such as RAM),non-volatile memory (such as ROM, flash memory, etc.) or any combinationthereof. System memory 720 can include one or more applications 722, andprogram data 724. Application 722 can include an algorithm 723 that isarranged to receive vehicle information and provide the vehicleinformation to a communication network or receive vehicle informationand determine estimate information, in accordance with the presentdisclosure. Program data 724 can include repair order data 725 thatcould be directed to any number of types of data. In some exampleembodiments, application 722 can be arranged to operate with programdata 724 on an operating system.

Computing device 700 can have additional features or functionality, andadditional interfaces to facilitate communications between the basicconfiguration 702 and any devices and interfaces. For example, datastorage devices 740 can be provided including removable storage devices742, non-removable storage devices 744, or a combination thereof.Examples of removable storage and non-removable storage devices includemagnetic disk devices such as flexible disk drives and hard-disk drives(HDD), optical disk drives such as compact disc (CD) drives or digitalversatile disk (DVD) drives, solid state drives (SSD), and tape drivesto name a few. Computer storage media can include volatile andnonvolatile, non-transitory, removable and non-removable mediaimplemented in any method or technology for storage of information, suchas computer-readable instructions, data structures, program modules, orother data.

System memory 720 and storage devices 740 are examples of computerstorage media. Computer storage media includes, but is not limited to,RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM,digital versatile disks (DVD) or other optical storage, magneticcassettes, magnetic tape, magnetic disk storage or other magneticstorage devices, or any other medium which can be used to store thedesired information and which can be accessed by computing device 700.Any such computer storage media can be part of device 700.

Computing device 700 can also include output interfaces 750 that caninclude a graphics processing unit 752, which can be configured tocommunicate to various external devices such as display devices 760 orspeakers via one or more A/V ports 754 or a communication interface 770.The communication interface 770 can include a network controller 772,which can be arranged to facilitate communications with one or moreother computing devices 780 over a network communication via one or morecommunication ports 774. The communication connection is one example ofa communication media. Communication media can be embodied bycomputer-readable instructions, data structures, program modules, orother data in a modulated data signal, such as a carrier wave or othertransport mechanism, and includes any information delivery media. Amodulated data signal can be a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal. By way of example, and not limitation, communicationmedia can include wired media such as a wired network or direct-wiredconnection, and wireless media such as acoustic, radio frequency (RF),infrared (IR) and other wireless media.

Computing device 700 can be implemented as a portion of a small-formfactor portable (or mobile) electronic device such as a cell phone, apersonal data assistant (PDA), a personal media player device, awireless web-watch device, a personal headset device, an applicationspecific device, or a hybrid device that include any of the abovefunctions. Computing device 700 can also be implemented as a personalcomputer, including both laptop computer and non-laptop computerconfigurations, or a server.

In some embodiments, the disclosed methods can be implemented ascomputer program instructions encoded on a non-transitorycomputer-readable storage media in a machine-readable format, or onother non-transitory media or articles of manufacture. FIG. 8 is aschematic illustrating a conceptual partial view of an example computerprogram product 800 that includes a computer program for executing acomputer process on a computing device, arranged according to at leastsome embodiments presented herein.

In one embodiment, the example computer program product 800 is providedusing a signal bearing medium 801. The signal bearing medium 801 caninclude one or more programming instructions 802 that, when executed byone or more processors can provide functionality or portions of thefunctionality described above with respect to FIGS. 1-7. In someexamples, the signal beating medium 801 can encompass acomputer-readable medium 803, such as, but not limited to, a hard diskdrive, a Compact Disc (CD), a Digital Video Disk (DVD), a digital tape,memory, etc. In some implementations, the signal bearing medium 801 canencompass a computer recordable medium 804, such as, but not limited to,memory, read/write (R/W) CDs, R/W DVDs, etc. In some implementations,the signal bearing medium 801 can encompass a communications medium 805,such as, but not limited to, a digital and/or an analog communicationmedium (e.g., a fiber optic cable, a waveguide, a wired communicationslink, a wireless communication link, etc.). Thus, for example, thesignal bearing medium 801 can be conveyed by a wireless form of thecommunications medium 805 (e.g., a wireless communications mediumconforming to the IEEE 802.11 standard or other transmission protocol).

The one or more programming instructions 802 can be, for example,computer executable and/or logic implemented instructions. In someexamples, a computing device such as the computing device 700 of FIG. 7can be configured to provide various operations, functions, or actionsin response to the programming instructions 802 conveyed to thecomputing device 700 by one or more of the computer-readable medium 803,the computer recordable medium 804, and/or the communications medium805.

It should be understood that arrangements described herein are forpurposes of example only. As such, those skilled in the art willappreciate that other arrangements and other elements (e.g. machines,interfaces, functions, orders, and groupings of functions, etc.) can beused instead, and some elements can be omitted altogether according tothe desired results. Further, many of the elements that are describedare functional entities that can be implemented as discrete ordistributed components or in conjunction with other components, in anysuitable combination and location.

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments will be apparent to those skilled in the art.The various aspects and embodiments disclosed herein are for purposes ofillustration and are not intended to be limiting, with the true scopebeing indicated by the following claims, along with the full scope ofequivalents to which such claims are entitled. It is also to beunderstood that the terminology used herein is for the purpose ofdescribing particular embodiments only, and is not intended to belimiting.

What is claimed is:
 1. A method comprising: receiving, from a first datasource that includes computer-readable service procedures, a firstservice procedure including a plurality of procedural steps forservicing a vehicle; identifying a procedural step of the plurality ofprocedural steps to supplement with supplemental service information;receiving, from a second data source that includes computer-readablevehicle information, information about vehicles sharing one or moreattributes with the vehicle; determining, based on the receivedinformation, a piece of supplemental service information to supplementthe identified procedural step; receiving image data indicative of thevehicle during performance of the first service procedure; detecting,based on the image data, an occurrence of a trigger event associatedwith performance of the identified procedural step of the first serviceprocedure; and in response to detecting the occurrence of the triggerevent, providing the piece of supplemental service information forpresentation on a display during performance of the identifiedprocedural step of the first service procedure.
 2. The method of claim1, wherein the image data comprises video data received from a camera.3. The method of claim 2, wherein the piece of supplemental serviceinformation is overlaid onto the video data as part of an augmentedreality (AR) presentation.
 4. The method of claim 2, wherein the displayis a portable display device which includes the camera.
 5. The method ofclaim 4, wherein the portable display device is a head-mountable device(HMD).
 6. The method of claim 1, wherein the trigger event comprises atechnician action associated with the identified procedural step of thefirst service procedure.
 7. The method of claim 1, further comprisingproviding a representation of the identified procedural step forpresentation on the display in response to detecting the trigger event.8. The method of claim 1, wherein identifying the procedural step tosupplement with supplemental service information comprises identifying avehicle component involved in the procedural step, and wherein the pieceof supplemental service information comprises a metric associated withthe vehicle component.
 9. The method of claim 8, further comprising:determining that the procedural step involves application of torque toat least one fastener of the vehicle component, and wherein the metricassociated with the vehicle component comprises at least one of a torquespecification or a torque pattern for the at least one fastener of thevehicle component.
 10. The method of claim 1, wherein the proceduralstep comprises a direction to add a fluid to one of a vehicle componentor a vehicle system, and wherein the piece of supplemental serviceinformation comprises at least one of a fluid capacity and fluidspecifications of the one of the vehicle component or the vehiclesystem.
 11. The method of claim 1, wherein identifying the proceduralstep to supplement with supplemental service information comprisesidentifying a vehicle component involved in the procedural step, andwherein the piece of supplemental service information comprises alocation in the vehicle of the vehicle component.
 12. The method ofclaim 1, further comprising: receiving usage information indicating ausage history of the vehicle; and determining the piece of supplementalservice information based at least in part on the usage information. 13.The method of claim 12, further comprising: identifying, based on thereceived usage information, at least one proximate vehicle component forinspection, wherein the at least one proximate vehicle component isproximate to a vehicle component involved in the first serviceprocedure, and wherein the piece of supplemental service informationincludes a notification to inspect the at least one proximate vehiclecomponent.
 14. The method of claim 13, further comprising: receivingdata indicative of a mileage limit for the at least one proximatevehicle component; and identifying the at least one proximate vehiclecomponent for inspection based on the receive4 d usage informationindicating a current vehicle mileage within a threshold amount of themileage limit for the at least one proximate vehicle component.
 15. Themethod of claim 1, further comprising receiving prior serviceinformation indicating a prior service history of the vehicle; anddetermining the piece of supplemental service information based at leastin part on the prior service information.
 16. The method of claim 1,further comprising: receiving at least one signal indicating anavailability of at least one vehicle part to be used in performance ofthe first service procedure, and wherein the piece of supplementalservice information includes an indication of the availability.
 17. Themethod of claim 1, further comprising: receiving a signal indicating acustomer approval of the procedural step, and wherein the piece ofsupplemental service information includes an indication of the customerapproval of the procedural step.
 18. A computing device, comprising: oneor more processors; and a non-transitory computer-readable data storagedevice storing computer-readable program instructions, wherein thecomputer-readable program instructions are executable by the one or moreprocessors to: receive, from a first data source that includescomputer-readable service procedures, a first service procedureincluding a plurality of procedural steps for servicing a vehicle;identify a procedural step of the plurality of procedural steps tosupplement with supplemental service information; receive, from a seconddata source that includes computer-readable vehicle information,information about vehicles sharing one or more attributes with thevehicle; determine, based on the received information, a piece ofsupplemental service information to supplement the identified proceduralstep; receive image data indicative of the vehicle during performance ofthe first service procedure; detect, based on the image data, anoccurrence of a trigger event associated with performance of theidentified procedural step of the first service procedure; and inresponse to detecting the occurrence of the trigger event, provide thepiece of supplemental service information for presentation on a displayduring performance of the identified procedural step of the firstservice procedure.
 19. The computing device of claim 18, wherein thecomputing device includes the display and a camera, wherein the imagedata comprises video data received from the camera, and wherein theprogram instructions are executable by the one or more processors tovirtually overlay the piece of supplemental service information onto thevideo data which is displayed on the display.
 20. A non-transitorycomputer-readable medium having stored therein instructions, that whenexecuted by a computing device, cause the computing device to performfunctions comprising: receiving, from a first data source that includescomputer-readable service procedures, a first service procedureincluding a plurality of procedural steps for servicing a vehicle;identifying a procedural step of the plurality of procedural steps tosupplement with supplemental service information; receiving, from asecond data source that includes computer-readable vehicle information,information about vehicles sharing one or more attributes with thevehicle; determining, based on the received information, a piece ofsupplemental service information to supplement the identified proceduralstep; receiving image data indicative of the vehicle during performanceof the first service procedure; detecting, based on the image data, anoccurrence of a trigger event associated with performance of theidentified procedural step of the first service procedure; and inresponse to detecting the occurrence of the trigger event, providing thepiece of supplemental service information for presentation on a displayduring performance of the identified procedural step of the firstservice procedure.